Skip to content

[WIP] feat: add ProposalDriver for CRD-based agent execution#232

Draft
rioloc wants to merge 2 commits into
lightspeed-core:mainfrom
rioloc:feat/subprocess-agent-driver
Draft

[WIP] feat: add ProposalDriver for CRD-based agent execution#232
rioloc wants to merge 2 commits into
lightspeed-core:mainfrom
rioloc:feat/subprocess-agent-driver

Conversation

@rioloc
Copy link
Copy Markdown
Collaborator

@rioloc rioloc commented May 8, 2026

⚠️ Review after #228 and #231 are merged — this PR builds on top of those changes.

Summary

Add SubprocessDriver — a new AgentDriver that manages OpenShift Proposal CR lifecycle via oc/kubectl CLI, enabling evaluation of agentic workloads running on the cluster.

The driver builds and applies Proposal CRs, polls .status.conditions until a terminal state is reached, handles auto-approval via ProposalApproval resources, and cleans up on completion. It works directly with CRD conditions (the stable API contract) rather than replicating the operator's internal phase derivation logic.

Design document: https://gist.github.com/rioloc/c4b29b534f02e46d0e023599c90ed0f0

Milestones

  • M1: ProposalDriver + Unit Tests — Driver implementation in isolation with full unit test coverage
    • ProposalDriver class with execute_turn, polling loop, auto-approve, cleanup
    • TerminalOutcome enum for driver-level terminal states
    • TurnData extended with proposal_spec, proposal_status, description, expected_proposal_status
    • Condition-based terminal detection (_is_terminal, _should_approve)
    • Unit tests for config validation, condition helpers, CR building, full lifecycle
  • M2: Integration with Framework — Wire into registry, constants, and agent config models
    • Register ProposalDriver in AgentDriverRegistry
    • Move ProposalAgentConfig to agents.py
    • Add "proposal" to SUPPORTED_AGENT_TYPES
    • Export from __init__.py
  • M3: Integration Tests — End-to-end tests against a live OpenShift cluster
    • Analysis-only and full lifecycle scenarios
    • Timeout handling, per-conversation agent override
    • Requires agentic operator deployed on cluster

Test plan

  • Unit tests pass: uv run pytest tests/unit/pipeline/evaluation/test_proposal_driver.py -v
  • Full test suite passes: make test
  • Quality checks pass: make pre-commit
  • Integration tests pass against live cluster (M3)

🤖 Generated with Claude Code

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 8, 2026

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 42f50b0d-0e55-4b2e-804a-f9ef7efeb553

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Comment thread src/lightspeed_evaluation/pipeline/evaluation/subprocess_driver.py Fixed
Comment thread src/lightspeed_evaluation/pipeline/evaluation/subprocess_driver.py Fixed
@rioloc rioloc force-pushed the feat/subprocess-agent-driver branch 3 times, most recently from 3af4e56 to 033af0d Compare May 15, 2026 09:14
import logging
import os
import shutil
import subprocess
Comment on lines +225 to +226
return subprocess.run(
[self._cli, *args],
@rioloc rioloc force-pushed the feat/subprocess-agent-driver branch 4 times, most recently from 09093ef to 08aa6d8 Compare May 19, 2026 15:52
@rioloc rioloc changed the title [WIP] feat: add SubprocessDriver for CRD-based agent execution [WIP] feat: add ProposalDriver for CRD-based agent execution May 20, 2026
@rioloc rioloc force-pushed the feat/subprocess-agent-driver branch from 08aa6d8 to b1271b6 Compare May 20, 2026 16:09
Introduce SubprocessDriver — a new AgentDriver that manages OpenShift
Proposal CR lifecycle via oc/kubectl CLI. The driver builds and applies
Proposal CRs, polls status conditions until terminal state, handles
auto-approval via ProposalApproval resources, and cleans up on completion.

Key design decisions:
- Works directly with CRD conditions (the stable API contract) instead
  of replicating the operator's internal DerivePhase() logic
- TerminalOutcome enum for driver-level terminal states
- TurnData extended with proposal_spec, proposal_status, description,
  and expected_proposal_status fields (backward-compatible, all Optional)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@rioloc rioloc force-pushed the feat/subprocess-agent-driver branch from b1271b6 to af6e041 Compare May 20, 2026 16:19
- Wire ProposalDriver into the evaluation pipeline: add "subprocess"
to SUPPORTED_AGENT_TYPES, move SubprocessAgentConfig to core/models,
register in AgentDriverRegistry via new registry module, and update
all exports. Fixes ProposalDriver.__init__ to accept enabled kwarg.
- Extend unit tests
- Add agentic integration tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@rioloc rioloc force-pushed the feat/subprocess-agent-driver branch from af6e041 to 1781c8b Compare May 20, 2026 16:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants